Sentiment analysis

To examine the correlation between market data and sentiment, this measure has to be extracted from every comment.

For calculation of the sentiment the package sentimentr It uses a dictionary based approach and also takes modifiers and negators into account, which is a major advantage in comparison to the package SentimentAnalysis

To consider the financial jargon used in these subreddits, the Loughran-McDonald dictionary is used for assigning polarity values to the words of a comment. The dictionary also has to be modified for the special slang of r/wallstreetbets

# adding and preparing the slang terms of wallstreetbets with their corresponding sentiment
wsb_specific_terms <- data.frame(x=c("to the moon", "cant go tits up", "tendies", "diamond hands", "paper hands"), y=c(1,1,1,1,-1)) %>% as_key()

# creating the modified polarity dictionary
modified_dict <- lexicon::hash_sentiment_loughran_mcdonald %>% 
  update_polarity_table(x=wsb_specific_terms)

# show a sample of positive and negative words of the dictionary
print(modified_dict %>% filter(y>0) %>% sample_n(10))

##                x y
##  1:      leading 1
##  2:     pleasant 1
##  3: achievements 1
##  4:   excitement 1
##  5:    delighted 1
##  6:   delightful 1
##  7:     premiere 1
##  8:  improvement 1
##  9:      vibrant 1
## 10:    enjoyable 1

print(modified_dict %>% filter(y<0) %>% sample_n(10))

##                 x  y
##  1:     subjected -1
##  2:     litigants -1
##  3:    tortuously -1
##  4:   misinformed -1
##  5: oversupplying -1
##  6:       purport -1
##  7:       ceasing -1
##  8:       outages -1
##  9:     depressed -1
## 10:  restatements -1

#wsb <- loadData("reddit", "wallstreetbets")
#stocks <- loadData("reddit", "stocks")
stocks <- readRDS("stocks.Rds")
wsb <- readRDS("wsb.Rds")

After adding the sentiment, we take a look at the the most positive and negative comments of both subreddits.

stocks %>% arrange(sentiment) %>% head() %>% select(comment, sentiment) %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

stocks %>% arrange(desc(sentiment)) %>% head() %>% select(comment, sentiment) %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

wsb %>% arrange(sentiment) %>% head() %>% select(comment, sentiment) %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

wsb %>% arrange(desc(sentiment)) %>% head() %>% select(comment, sentiment) %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

Since we are only interested in if a stock is positive, negative or neutral, the values are recoded to 1, -1 and 0 respectively.

wsb <- wsb %>% mutate(sentiment = case_when(sentiment > 0 ~ 1,
                                            sentiment < 0 ~ -1,
                                            sentiment == 0 ~ 0))
stocks <- stocks %>% mutate(sentiment = case_when(sentiment > 0 ~ 1,
                                            sentiment < 0 ~ -1,
                                            sentiment == 0 ~ 0))

Ticker Extraction

To find tickers discussed in a comment, the corresponding symbols have to be extracted.

Two approaches to achieve have been tried:

1. Regex-based extraction:

A regular expression can be used to find ticker mentions inside a comment. Ticker symbols are identified by assuming they are a two to four upper case character long string preceeded and followed by zero or more non-alphanumeric characters, e.g. " GME." or " CDE".

stringr::str_match_all(c("I bought AMC and BB", "I sold Palantir. PLTR was not the best stock to invest into"), '[^A-Za-z0-9]+([A-Z]{2,4})[^A-Za-z0-9]*')

## [[1]]
##      [,1]    [,2] 
## [1,] " AMC " "AMC"
## [2,] " BB"   "BB" 
## 
## [[2]]
##      [,1]      [,2]  
## [1,] ". PLTR " "PLTR"

Disadvantages of this method are:

• False positives like “WSB” for Wallstreetbets or “ETF” for Exchange-traded fund. A remedy to this could an expansion of the regex or a subsequent filter on the most mentioned false positives.
• Actual company names are not recognized, so if “Palantir” gets mentioned, it is not detected as PLTR.

These disadvantages can be remedied by using a dictionary-based approach.

2. Dictionary-based extraction:

For this approach a list of the top 100 actively traded stocks traded on the NYSE is used.

nyse_lst <- read.csv("nyse_lst.csv")
nyse_lst %>% 
  head() %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

The names of the companies are also modified, words and Phrases like “Ltd.”, “Company”, “Group”, “Share” have been removed manually to capture the informally used phrases one would use to refer to these companies. In addition, tickers consisting of only one character are replaced with this modified name to avoid false matches, e.g. “F” for “Ford”.

nyse_lst_mod <- read.csv("nyse_lst_modified.csv")
nyse_lst_mod %>%
  head() %>%
  kbl() %>%
  kable_material(c("hover")) %>%
  kable_styling(full_width = TRUE)

To add this information, a helper function was written which adds columns for sentiment and the mentioned tickers respectively.

source("ticker_extraction.R")
example_df <- data.frame(comment = c("I bought AMC and BB", "I sold Palantir. PLTR was not the best stock to invest into but GME was"), user = c("A", "B"))

add_tickers_sentiment(example_df,
                      ticker_list = nyse_lst_mod)$ticker

## [1] 1

## [[1]]
## [1] "AMC" "BB" 
## 
## [[2]]
## [1] "PLTR" "GME"

Using this approach, the ticker symbols as well as the names of the companies are extracted. In addition, false positives are avoided by only extracting symbols mentioned in the modified list.

Community detection

For the community detection we used the library igraph. For this purpose the subreddit’s data was transformed into a graph representation. Nodes represent the users of the subreddit, while interaction between the users is represented by edges. The interactions are classified into:

1. Answers to a post: direct comments to a post. For this case an edge is created between the user and the author of the post.

2. Answer to a comment: comments made as answer to other comments. For this case an edge is created between two users and it requires to use the hierarchy of comments to find the proper connection.

The weight of the edges is given by the number of interactions between the two users. A visual representation of the graph created by the comments between the dates 2021-01-23 and 2021-01-26 for the subreddit is presented in the following plot.

## `summarise()` has grouped output by 'from'. You can override using the `.groups` argument.

Clustering Algorithms:

For the community detection we considered three of the available algorithms in the igraph library, they are:

• Louvain: Algorithm based on modularity. It builds a hierarchy of communities while maximizing the modularity value.
• Infomap: Algorithm based on the Map equation.
• label propagation: The algorithm starts by given a unique labels to each node, and it propagates iteratively the labels through the network by updating the neighbors until no node need to update the label.

Since there is no external information about the communities detected, modularity is used as main measure to evaluate the quality of the clustering. Modularity measures how strong is the community structure generated and it can be defined as “a normalized tradeoff between edges covered by clusters and squared cluster degree sums”[5].

In order to compare the cluster algorithms, we evaluate all of algorithms over 4 different samples of the data as shown in the following table. We used both subreddits wallstreetbets and stocks with different sample sizes. With this test we recorded the modularity and execution time. The results from the evaluation are given in the following plots.

evaluations <- data.frame(
  test = 1:4,
  collectionName = c("wallstreetbets", "stocks", "wallstreetbets", "stocks"),
  initial_time = c("2021-02-03", "2020-11-03", "2021-01-03", "2021-01-03"),
  end_time = c("2021-02-05", "2020-11-07", "2021-01-07", "2021-01-13"),
  number_comments = rep(NA, 4)
)

result <- data.frame()
for (i in 1:nrow(evaluations)) {
  row <- evaluations[i, ]
  if(row$collectionName =="wallstreetbets"){
    eval_df <-  wsb %>%
      filter(comm_date>=row$initial_time, comm_date<=row$end_time)
  }else{
    eval_df <-  stocks %>%
      filter(comm_date>=row$initial_time, comm_date<=row$end_time)
  }
  result_evaluation <-
    clustering_evaluation(row$test,
                          "reddit",
                          row$collectionName,
                          row$initial_time,
                          row$end_time,
                          eval_df)
  #update number of comments per test
  evaluations[[i, "number_comments"]] <-
    result_evaluation$number_comments
  result <- rbind(result, result_evaluation$df)
}

kbl(evaluations,
    col.names = c("Test",
                  "Collection Name",
                  "Initial Date", 
                  "Final Date",
                  "Sample size"),
    caption = "Test samples")

modularity_clusters <- result %>%
  filter(measure == "modularity") 

ggplotly(ggplot(data=modularity_clusters, aes(x=id_test))+
           geom_bar(aes(y=value,fill=algorithm), 
                    stat = "identity", position = "dodge")+
           labs( x = "Test",
                 y = "Modularity",
                 title = "Modularity per algorithm for each test"))

In terms of modularity, Louvain performed better than Infomap and Label Propagation. The highest possible value for modularity is 1.0 when all of the clusters are disconnected subgraphs. The values that Louvain achieved are higher than 0.55 which represents a strong community structure.

When we evaluate the execution time for the different algorithms, Louvain outperfoms Infomap but the execution time is very similar between Louvain and Label propagation.

In addition to the previous test in separated samples, we assessed the robustness of Louvain using the library robin [6] for the whole data for each subreddit.

To test the goodness of the communities detected by Louvain in the each complete subreddit we followed the first workflow from robin, where the stability of the algorithm is compared against random perturbations of the original graph. The stability measure we chose was VI metric. The results from this evaluation are in the next plots.

Stocks subreddit

Wallstreetbets subreddit

For both datasets the stability measure curves for Louvain compared to the null model are very close. For the stocks subreddit the AUC values we got were 0.2631427 for the real data and 0.2494959 for the null model and for the wallstreetbets subreddit were 0.2357264 for the real data and 0.2471575 for the null model. Considering also the p-values on the right side of the plots, we can conclude that the results from the Louvain community detection are statistically significant, which means that is a good algorithm for this dataset. For Wallstreetbets it is statistically significant just for perturbation lower than 0.3, which is not so ideal.

When we compare with robin the best two algorithms from our modularity test (Louvain and Label Propagation) is possible to confirm the previous findings. For both subreddits Louvain outperforms the Label Propagation algorithm.

Considering these tests, we selected Louvain as the clustering algorithm for the rest of the analysis in our project. For the previous graph representation of the data, the following communities were created with Louvain algorithm.

## `summarise()` has grouped output by 'from'. You can override using the `.groups` argument.

To analyze the content of the comments for each cluster, we inspect some random clusters to find the most common words, which are displayed in the following wordclouds. For some clusters we can find that they use quite often the same words like “stocks” and “buy”, but we also observed that we have different topics on each cluster.

par(mfrow=c(2,2),
    oma = c(0,0,0,0),
    mar = c(0,0,0,0))
get_word_cloud_community(posts_stocks,communities(lc),2)
get_word_cloud_community(posts_stocks,communities(lc),5)
get_word_cloud_community(posts_stocks,communities(lc),8)
get_word_cloud_community(posts_stocks,communities(lc),10)